The phrase "ugly side" refers to the negative or unpleasant aspects of something or someone. It represents the unappealing or unpleasant qualities that are not typically seen or acknowledged.